Algorithmic Concept-Based Explainable Reasoning

Recent research on graph neural network (GNN) models successfully applied GNNs to classical algorithms and combinatorial optimisation problems. This has numerous benefits, such as allowing applications of when preconditions are not satisfied, or reusing learned sufficient training data is available can't be generated. Unfortunately, a key hindrance these approaches their lack explainability, since black-box that cannot interpreted directly. In this work, we address limitation by applying existing work concept-based explanations GNN models. We introduce concept-bottleneck GNNs, which rely modification the readout mechanism. Using three case studies demonstrate that: (i) our proposed model capable accurately learning concepts extracting propositional formulas based for each target class; (ii) achieve comparative performance with state-of-the-art models; (iii) can derive global concepts, without explicitly providing any supervision graph-level concepts.